|
The Flajolet–Martin algorithm is an algorithm for approximating the number of distinct elements in a stream with a single pass and space-consumption which is logarithmic in the maximum number of possible distinct elements in the stream. The algorithm was introduced by Philippe Flajolet and G. Nigel Martin in their 1984 paper "Probabilistic Counting Algorithms for Data Base Applications". Later it has been refined in the papers "LogLog counting of large cardinalities" by Marianne Durand and Philippe Flajolet, and "HyperLogLog: The analysis of a near-optimal cardinality estimation algorithm" by Philippe Flajolet et al. In their 2010 paper "An optimal algorithm for the distinct elements problem", Daniel M. Kane, Jelani Nelson and David P. Woodruff gives an improved algorithm which uses nearly optimal space, and has optimal ''O''(1) update and reporting times. ==The algorithm== Assume that we are given a hash function which maps input to integers in the range and where the outputs are sufficiently uniformly distributed. Note that the set of integers from 0 to corresponds to the set of binary strings of length . For any non-negative integer , define to be the -th bit in the binary representation of , such that: We then define a function which outputs the position of the least significant 1-bit in the binary representation of : where . Note that with the above definition we are using 0-indexing for the positions. For example, since the least significant bit is a 1, and since the least significant 1-bit is at the third position. At this point, note that under the assumption that the output of our hash-function is uniformly distributed, then the probability of observing a hash-output ending with (a one, followed by zeroes) is since this corresponds to flipping heads and then a tail with a fair coin. Now the Flajolet–Martin algorithm for estimating the cardinality of a multiset is as follows: # Initialize a bit-vector BITMAP to be of length and contain all 0's. # For each element in : ## index = . ## . # Let denote the largest index such that . # Estimate the cardinality of as where . The idea is that if is the number of distinct elements in the multiset , then is accessed approximately times, is accessed approximately times and so on. Consequently if then is almost certainly 0, and if then is almost certainly 1. If then can be expected to be either 1 or 0. The correction factor is found by calculations which can be found in the original paper. 抄文引用元・出典: フリー百科事典『 ウィキペディア(Wikipedia)』 ■ウィキペディアで「Flajolet–Martin algorithm」の詳細全文を読む スポンサード リンク
|